Skip to content

feat(demo): run real CI on the github rung - #604

Merged
behinddwalls merged 1 commit into
mainfrom
preetam/github-actions-runner
Aug 18, 2026
Merged

feat(demo): run real CI on the github rung#604
behinddwalls merged 1 commit into
mainfrom
preetam/github-actions-runner

Conversation

@behinddwalls

Copy link
Copy Markdown
Collaborator

Summary

Why?

The GitHub rung was described as the one where everything is real, and it was not. It took real pull requests from a real repository and pushed real commits back — but the build in the middle was the fake runner, passing instantly. landed there meant the queue had run its pipeline, not that anything had been built.

That is a bad thing to be unclear about, because it is the one rung a reader is most likely to trust. Watching three pull requests merge in twenty seconds looks like a submit queue validating and landing them; nothing in the output says the validation was a stub.

It was also the last thing separating the demo from the product. Speculation exists to test the combination — a batch on top of the in-flight changes assumed to land before it — and a fake runner exercises the trigger-and-poll loop while proving nothing about the combination it was built to check.

The switch was already documented as an opt-in, three manual steps in a section further down. Nobody who has just watched the demo will go and do them, which meant the interesting half of the feature was reachable in principle and unused in practice.

What?

demo/provider/github/profiles.yaml selects the githubactions runner for demo-queue instead of inheriting the fake one. That is the whole change in behaviour: the queue now dispatches a workflow_dispatch per speculative batch, polls the resulting check, and decides merge or eject on the answer.

The defaults are untouched, so a queue not listed still cannot reach GitHub by accident. fake and git are untouched: their builds stay instant, which is what keeps the first two rungs free and fast.

The docs stop describing this as opt-in. "Using real CI" now covers what the repository must provide rather than what to switch on — a dispatchable workflow accepting the four sq_* inputs, and, the part that decides whether any of it means anything, a workflow that actually applies both input sets. A workflow that checks out the pull request and builds it is green about the wrong thing.

Two consequences are stated where a reader meets them rather than buried: a land now takes as long as the workflow does, and every speculation path dispatched is a billable run — which makes the queue's build budget a spending control as much as a CI one.

The ladder table gains a Building it column. It previously described where changes come from and what landing does, and said nothing about the build, which is how the fake runner stayed invisible in the summary a reader skims. The same paragraph now also says plainly that fake and git fake the change provider too, so sq-files= is how the analyzer sees paths there — true before this change and undocumented.

Token permissions move Actions from "only if you switch the build runner" to required, and the classic-PAT line from "repo, plus workflow if…" to both.

Test Plan

  • ✅ ran it: PROVIDER=github make demo-requests COUNT=2 against a scratch repository opened pull requests 525 and 526 and landed both
  • ✅ the dispatches are real and are the queue's, confirmed from the Actions API rather than from the queue's own logs: two new workflow_dispatch runs at 18:53:26, both success, titled SQ - demo-queue - speculative batch — the workflow's own run-name, built from the sq_queue input the runner sent
  • ✅ before this change the same repository showed no workflow_dispatch run since 2026-08-10 and zero check-runs on a landed head, which is what established the gap in the first place
  • make fmt, make lint

Not verified: cancellation. The runner cancels a build whose speculation path is abandoned, but the workflow used here finishes in seconds, so a run is almost always complete before there is anything to cancel — the same reason abandoned-path builds show up as post-terminal events in a fake run. It needs a slow workflow to exercise honestly.

A note for anyone pointing this at their own repository: ci.yml and behinddwalls/sq-demo are marked in the file the same way merge.yaml's remote is. A workflow triggered only by pull_request cannot be dispatched at all, and GitHub rejects the call rather than falling back.

@behinddwalls
behinddwalls requested review from a team and sbalabanov as code owners August 16, 2026 18:56
@behinddwalls
behinddwalls force-pushed the preetam/github-actions-runner branch from 6a563c2 to a6eedd2 Compare August 18, 2026 15:25
@behinddwalls
behinddwalls force-pushed the preetam/github-actions-runner branch from a6eedd2 to 886e79b Compare August 18, 2026 16:28
@behinddwalls
behinddwalls force-pushed the preetam/github-actions-runner branch from 886e79b to 060ae51 Compare August 18, 2026 16:44
@behinddwalls
behinddwalls force-pushed the preetam/github-actions-runner branch from 060ae51 to a24ade0 Compare August 18, 2026 16:55
@behinddwalls
behinddwalls force-pushed the preetam/github-actions-runner branch from a24ade0 to c2df7d4 Compare August 18, 2026 17:15
Base automatically changed from preetam/quickstart-audit to main August 18, 2026 17:23
## Summary

### Why?

The GitHub rung was described as the one where everything is real, and it was not. It took real pull requests from a real repository and pushed real commits back — but the build in the middle was the fake runner, passing instantly. `landed` there meant the queue had run its pipeline, not that anything had been built.

That is a bad thing to be unclear about, because it is the one rung a reader is most likely to trust. Watching three pull requests merge in twenty seconds looks like a submit queue validating and landing them; nothing in the output says the validation was a stub.

It was also the last thing separating the demo from the product. Speculation exists to test the *combination* — a batch on top of the in-flight changes assumed to land before it — and a fake runner exercises the trigger-and-poll loop while proving nothing about the combination it was built to check.

The switch was already documented as an opt-in, three manual steps in a section further down. Nobody who has just watched the demo will go and do them, which meant the interesting half of the feature was reachable in principle and unused in practice.

### What?

`demo/provider/github/profiles.yaml` selects the `githubactions` runner for `demo-queue` instead of inheriting the fake one. That is the whole change in behaviour: the queue now dispatches a `workflow_dispatch` per speculative batch, polls the resulting check, and decides merge or eject on the answer.

The defaults are untouched, so a queue not listed still cannot reach GitHub by accident. `fake` and `git` are untouched: their builds stay instant, which is what keeps the first two rungs free and fast.

The docs stop describing this as opt-in. "Using real CI" now covers what the *repository* must provide rather than what to switch on — a dispatchable workflow accepting the four `sq_*` inputs, and, the part that decides whether any of it means anything, a workflow that actually applies both input sets. A workflow that checks out the pull request and builds it is green about the wrong thing.

Two consequences are stated where a reader meets them rather than buried: a land now takes as long as the workflow does, and every speculation path dispatched is a billable run — which makes the queue's build budget a spending control as much as a CI one.

The ladder table gains a **Building it** column. It previously described where changes come from and what landing does, and said nothing about the build, which is how the fake runner stayed invisible in the summary a reader skims. The same paragraph now also says plainly that `fake` and `git` fake the *change provider* too, so `sq-files=` is how the analyzer sees paths there — true before this change and undocumented.

Token permissions move Actions from "only if you switch the build runner" to required, and the classic-PAT line from "`repo`, plus `workflow` if…" to both.

## Test Plan

- ✅ ran it: `PROVIDER=github make demo-requests COUNT=2` against a scratch repository opened pull requests 525 and 526 and landed both
- ✅ the dispatches are real and are the queue's, confirmed from the Actions API rather than from the queue's own logs: two new `workflow_dispatch` runs at 18:53:26, both `success`, titled `SQ - demo-queue - speculative batch` — the workflow's own `run-name`, built from the `sq_queue` input the runner sent
- ✅ before this change the same repository showed no `workflow_dispatch` run since 2026-08-10 and zero check-runs on a landed head, which is what established the gap in the first place
- ✅ `make fmt`, `make lint`

Not verified: cancellation. The runner cancels a build whose speculation path is abandoned, but the workflow used here finishes in seconds, so a run is almost always complete before there is anything to cancel — the same reason abandoned-path builds show up as post-terminal events in a fake run. It needs a slow workflow to exercise honestly.

A note for anyone pointing this at their own repository: `ci.yml` and `behinddwalls/sq-demo` are marked in the file the same way `merge.yaml`'s remote is. A workflow triggered only by `pull_request` cannot be dispatched at all, and GitHub rejects the call rather than falling back.
@behinddwalls
behinddwalls force-pushed the preetam/github-actions-runner branch from c2df7d4 to 4f07326 Compare August 18, 2026 17:23
@behinddwalls
behinddwalls added this pull request to the merge queue Aug 18, 2026
Merged via the queue into main with commit 5750f90 Aug 18, 2026
15 checks passed
@behinddwalls
behinddwalls deployed to stack-rebase August 18, 2026 17:27 — with GitHub Actions Active
@behinddwalls
behinddwalls deleted the preetam/github-actions-runner branch August 18, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants